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Abstract 



A redundant generating function is a generating function having terms which are 
not part of the solution of the original problem. We use redundant generating functions 
^_^ I to study two path problems. In the first application we explain a surprising occurrence 

V,^ ■ of Catalan numbers in counting paths that stay below the line y = 2x. In the second 

^ ■ application we prove a conjecture of Niederhausen and Sullivan. 



1 Introduction 



^ ■ One method for solving recurrences with boundary conditions is extending the region in 

O . which the recurrence is satisfied. Then the generating function for the extended recurrence 

^yQ. in the new region may become simpler. Following MacMahon [4, pp. 128], we call the 

cn I generating function a redundant generating function, since it contains some terms which 

C^ I are not part of the solution of the original problem. This method is especially useful in 

lattice path counting problems. In this paper we first review the redundant generating 
function for ballot numbers. Then we study variations of the ballot problem. In one of 
these variations, we explain a surprising occurrence of Catalan numbers. When we define 
"k>( I the number D'^im — n, n) for m, n > by 



oo oo 

J2 D'^i^ - n, n)x"'r = (l + 5]](-l)"+^C„t"+^) (l - x{l + t) 



m,n=0 n=0 



where C„ is the nth Catalan number [7, pp. 219-229] (A000108), we will see that D'2{m,n) 
is the number of lattice paths from (0,0) to {m,n) (where < n < 2m), with unit up (f) 
steps (0,1) and unit right (— ;•) steps (1,0), that never cross the line y = 2x. There is no 
combinatorial significance for D'2{m,n) where 2m <n ot n <0. 

Next, we prove a conjecture [1] of Niederhausen and Sullivan using a redundant generating 
function. When we define the number S'{m,n) for ?n, n > by 

m,n=0 



the conjecture says that for 2m > n, S'{m,n) is the number of paths from (0,0) to {4m — 
n — 1,2m — n + 1), with up (/^) steps (1, 1) and down (\) steps (1, —1), that avoid four 
consecutive up {/^) steps and never go below the x-axis. There is no combinatorial meaning 
for S'{m,,n), where 2m, < n. Finally, we will find a redundant generating function for 
the numbers counting lattice paths with arbitrary given up steps, but one down step. In 
Theorem 5.1 a redundant generating function is given that generalizes the previous two 
redundant generating functions (1) and (2). 

2 The ballot problem 

Let us consider the ballot problem [5, pp. 1-8] that asks about the number B{m,, n) of lattice 
paths from (1,0) to {m.,n) (where m > n), with unit up (t) steps (0, 1) and unit right (— )■) 
steps (1, 0), that stay below the line x = y. The number B{m,, n) can easily be computed by 
the recurrence 

B{m, n) = B{m — 1, n) + B{m^ n — 1) for < n < ?7i, {m, n) ^ (1, 0) (3) 

with the initial condition S(1,0) = 1 and the boundary conditions B{m^—1) = and 
B{m,,n) = for < m, < n. The recurrence (3) holds since a path ending at {m,,n) can be 
obtained from either a path ending at {m, — 1, n) followed by a unit right (->) step (1, 0) or 
a path ending at (m, n — 1) followed by a unit up (t) step (0, 1). 
Let c{x) be the Catalan number generating function 



C{X} - 2_^L.nX - — , 

n=0 



where C„ = -tt i'^^') is the nth Catalan number. 

" n+l \ n / 

Now we want to find the generating function 2_. B{m,,n)x"^y^. From the well-known 

m>n>0 

fact that B{n + 1, n) = C„, we can use a variation of (3) which is 

( 1 if (m,n) = (1,0) 

B{m, n) — B{m — 1, n) — B{m, n — 1) = I — Cm-i if m = n (4) 

[^ otherwise. 

Then the recurrence (4) is valid for all m, n > 0, and is also easy to see combinatorially. 
Multiplying the recurrence (4) by x™?/" and summing on m and n, we get 

{1 — X — y) 2_, B{m,,n)x"^y" = X — xyc{xy). 





m>n>0 






So, we have 










^ B{m,n)x'^y" = 

m>n>0 
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This is equivalent to 



Table 1: The values of B(m,n) 



1+ y B(m,n)x"'y''= 

^-^ \ — XL 



m>n>0 



xc{xy) 



(5) 



Identity (5) can also be proved combinatorially by using the prime decomposition for Dyck 
paths [3, pp. 1027-1030]. 

Instead of taking B{m,n) = for m < n, we could try to define B{m,n) everywhere in 
the first quadrant so that the recurrence (3) is satisfied as much as possible. We can do this 
by rewriting the recurrence (3) as 

Bim — l,n) = B(m, n) — Bim, n — 1) for m < n. 



with B^m, n) = for all m < 0. So let us define B{m, n) to be B{m, n) for m > n >0, and 

B^m — 1, n) = B^m, n) — B{m, n — 1) ioi m < n, (6) 

with B{m, n) = for all m < 0. Table 1 shows the values of B{m, n) for the first quadrant. It 
is easy to see that Blm, n) can be extended to the whole first quadrant so that the recurrence 
is satisfied everywhere except (1, 0) and (0, n) forn G P = {1, 2, 3, . . .}. In fact, the recurrence 
(6) is satisfied everywhere except at (1,0) and (0,1). To see this, we define B'{m,n) by 
B'{1, 0) = 1, B'{0, 1) = -1, 5'(m, -1) = 0, and B'{-l,m) = for all m G N = {0, 1,2,.. .} 
with the recurrence for all {m^n) G N^, {m,n) ^ (1,0) or {m^n) ^ (0, 1), 

B'{m, n) = B'{m - 1, n) + B'{m, n-l). 

Then it is easy to show by induction that B'{'m,n) = —B'{n,m) for all m,n & N, so we 
have that B'{m, m) = for all ?7i G N. Therefore, the values of B^m, n) and B'^m, n) for all 
{m,n) G N^ coincide. 

From this fact, we can assume that the recurrence for B'{m, n) for the first quadrant is 



B'im, n) — B'(m — 1, n) — B'im, n — 11 



1 if (m,n) = (1,0) 
-1 if (m,n) = (0,1) 
otherwise, 



(7) 



with the boundary conditions B'{m^ n) = for ?7i < or n < 0. 
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Table 2: The values of C2{m,n) 

In terms of generating functions, the recurrence (7) is equivalent to the equation 

{1 — X — y) y^ B'{m,n)x"^y^ = X — y, 



rra,n>0 

which is the same as 



J2 B\m,n)x'^y- = - 



x-y 



X — y 

m,n>0 ^ 



Expanding in powers in x and y and equating coefficients of x'^y^ in equation (8), we have 
the ballot number formula: 

, , f'm + n — l\ /?7i + n — 1\ m — n fm + n 

B (m,n 



771 — 1 J \ 771 J 771 + 71 \ 771 

which satisfies the recurrence (3) with the initial and boundary conditions. That is, 

B{7n, n) = B'{7n, n) for all < n < ?7i. 

Also, we can see that the redundant generating function {x — y)/{l — x — y) for B'{m, n) is 
much simpler than the generating function x(l — yc[xy)^ / {1 — x — y) for B{m, n). 

3 Variations of the ballot problem 

Now let us consider [2] the line x = py, where p G P, as the boundary line in the ballot 
problem instead of the boundary line x = y. Then this problem asks about the number 
Cp{7n, n) of lattice paths from (1, 0) to (m, n), (where m > pn) with unit up (t) steps (0, 1) 
and unit right (— ;■) steps (1,0), that never touch the line x = py. Table 2 shows the values 
of C2{7n, 7i) in the case p = 2. 

If we extend Cp{m, n) to the first quadrant then the recurrence seems to be satisfied 
everywhere except at (1, 0) and (0, 1). More precisely, let us define values of C Am., n) by the 
recurrence 

C;(m,n)=C;(m-l,n) + C;(m,n-l), 

and the initial conditions Cp(l, 0) = 1, Cp(0, 1) = — p, and the boundary conditions Cp{7n, —1) 
and Cp(— 1, m) = for all m G N. This is equivalent to 

1 if (m,n) = (1,0) 
C;(m,n)-C;(m-l,n)-C;(m,n-l)= <( -p if (m, n) = (0, 1) (9) 

otherwise, 
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Table 3: The values of D2(m, n) for < n < 2m 



with the boundary conditions C'{m,n) = for m < or n < 0. The recurrence (9) is vahd 
for all 712,71 > 0. 

To show that C'p{m, n) = Cp{m, n) for m > pn, it is enough to show that C'p{pn, n) = 
for all n E N. Multiplying the recurrence (9) by x'^y^ and summing on m and n, we get 



Y, C'p{m, n)x^y- = - 



m,n>0 



X — py 
-x-y 



(10) 



Expanding in powers of x and y and equating coefficients of x™?/"' in equation (10), we have 
the formula: for all m,n Efi 



C'{m,n) 



m + n—l\ fm + n — l\ m — pnfm + n 
p 



m — 1 



m 



m + n \ m 



From the formula for C'p{m,n), it is easy to see that Cp{pn,n) = 0. Therefore the values of 
Cp(m,n) are the same as the values of Cp{m,n) = since C'p{pn,n) = and Cp{pn,n) = 
for all n e M. 

At this point it might be natural to ask what happens if we use the line y = px (where 
p G P) as the boundary line instead of the line y = x. For convenience' sake we allow a 
lattice path to touch the line y = px. Let Dp(m, n) be the number of lattice paths from (0, 0) 
to {m,n) (where n < pm), with unit up (t) steps (0, 1) and unit right (— )■) steps (1,0), that 
never cross the line y = px. Let us consider the case p = 2. Since the slope of the boundary 
line is 2, we require D2{'m, 2m + 1) = and D2{m, 2m + 2) = for all m E "N. Then the 
number D2{m,,n) can easily be computed by the recurrence 

D2{m,, n) = D2{m, — 1, n) + D2{m,, n — 1) for < n < 2m, {m, n) ^ (0, 0), 

with the initial condition -D2(0, 0) = 1 and the boundary conditions 

D2{m, 2m + 1) = 0, D2{m, 2m + 2) = 0, and D2{m, -1) = for all m^n. (11) 

Table 3 shows the values of D2{m,,n). From the table we can observe that the numbers 
on the line y = 2x are the same as the numbers (A001764) on the line x — 1 = 2?/ in Table 2, 
but the other numbers off the line y = 2x are different from the numbers in Table 2. We can 
prove that the numbers on the hne y = 2x in Table 3 are the same as the numbers on the 
hne X — 1 = 2y in Table 2 by finding a bijection between the set of lattice paths from (0, 0) 



to {pn,n), with unit up (t) steps (0, 1) and unit right (—!■) steps (1, 0), that never cross the 
hne X = py and the set of lattice paths from (0,0) to {n,pn), with unit up (t) steps (0, 1) 
and unit right (— ;■) steps (1, 0), that never cross the hne y = px. 

Now let us present the bijection. When we are given a lattice path P from (0, 0) to 
(pn, ra), first reverse it to obtain a path P'' from (pn, n) to (0, 0) consisting of down (J,) steps 
and left (^) steps and change a down (|) step with a right (— )■) step and a left (^) step 
with a up (t) step. Then we have a lattice path P' from (0, 0) to {n,pn). In the same way, 
we can transform the lattice path P' from (0, 0) to {n,pn) into the lattice path P from (0, 0) 
to {pn,n). Since the path P does not cross the line y = px, we know that for each i, the 
number of left steps among the first i steps in the path P^ is always less than or equal to 
p times the number of down steps among the first i steps. So the corresponding path P' to 
the path P is a path that never crosses the line y = px, and vice versa. Therefore we have 

Dp{n,pn) = Cpipn + 1, n) for n > and p > 1. 

From the path transformation, we can also derive the number of lattice paths starting at 
a point that is below the line y = px and ending on the line y = px, with unit up (t) steps 
(0, 1) and unit right (— )■) steps (1, 0), that never cross the line y = px. That is, the number 
of lattice paths from {i,j) to {n,pn) (where j < pi) having n — i unit right (— )■) steps and 
pn — j unit up (t) steps, that never cross the line y = px is Cp(j)n — j + 1, n — z) . 

3.1 Another way to get Catalan numbers 

If we compute, as before, the values of D2{m,n) using the recurrence 

D2{m — l,n) = D2{m, n) — D2{m, n — 1) (12) 

with the boundary conditions (11) extended to the first quadrant, then we do not find 
anything interesting. So let us extend these values of D2{m, n) satisfying the recurrence (12) 
to the region {{x,y) ^ 7? \ y > —x,y > 0}. There is no combinatorial significance for 
D2{m,n) where n > 2m. 

Table 4 shows the values of D2{m, n). Note that the numbers on the line x = —2 in Table 
4 are (— 1)"M„, where M„ is the nth Motzkin number (A001006). Surprisingly, we find that 
the number D2{—n — 1, n + 1) is equal to {—l)"'~^^Cn for n G N on the boundary line y = —x 
in the table. 

Before we find the redundant generating function for D2{m,n) (We will give it after 
Theorem 5.1) let us prove that we have the Catalan numbers on the boundary line y = —x 
in Table 4. Since D2{n, 2n) = C2(2n + 1, n) for all n G N, we have 

D2in,2n) 



2n + l\n 

So, by the recurrence (12) with D2{n, 2n + 1) = 0, we have 

1 /3n\ 

D2(n-l,2n + l) = -- r for all n G N. 
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Table 4: The values of D2{m, n) 

For each n &'H, let us define a^ to be D2{n — l,2n + 1) and 6„ to be D2{—n — l,n + 1). 
Now let us find a formula for 6„ in terms of ai in Table 4. As shown in Figure 1, we can see 

D2{m — l,n) = D2{m, n) — D2{m, n — 1). 

By iterating this recurrence, we can easily derive that 6„ = ^^o(~-'-)"~^*''™^«' where c„j is 
the number of paths from {i — l,2i + 1) to (— n — 1, n + 1) with steps (—1, 0) and (—1, 1). 
That is, 

^3i + n — 2i\ fn + i^ 

32 y V 3z 
The formula hn = (— 1)"^^C„ is a consequence of the following lemma. 



64(^^-9^-6^-4. 



3)a2 




69r-2V^-l-± — l-i(-l 



Ol 



h(T)^ 
Figure 1: The values of D2{m,n) 



Lemma 3.1. For n > 0, 



i=0 



n + i 
3i }2i 



n 



f2n 
TV n 



Proof. Since the binomial coefficient ('^n^*) is nonzero only for < i < \n/2\ , the summation 
i on the left is in that range. So, for n > 

^ /n + A 1 /3A _ l^^ (n + 2)! 1 (3^)! 



X 3z J2i + l\ij ^ {3t)\{n-2i)\ 2z + 1 z! (2z)! 



Ln/2J 

E 

Ln/2J 

E 

j=0 



(n + i)\ 



{n-2i)\i\{2i + l)\ 



{n-2i)\i\ n\{2i + l)\ 



^ oo 

+1^ 



n + i\ I n + 1 



n + 1 ^^-^ \ n I \n — 2i 
■1=0 ^ ^ ^ 

2n 



n + l\n 

where the second last equality is only valid for < i < [?^/2j and the last equality is derived 
from the fact that the coefficient of x" in ((1 + x)/(l — a;^)) is the same as the coefficient 
of x" in l/(l-x)"+^ D 

Note that in the paper [8] Sun proved Lemma 3.1 combinatorially by using binary and 
ternary trees. Another proof of Lemma 3.1 is in the paper [6]. 
So, we have the following result. 

Theorem 3.1. For each n G N, the number D2{—n — 1, n + 1) is equal to {—l)^~^^Cn, where 
Cn is the nth Catalan number. 

Note that there is no combinatorial significance for D2{—n — 1, n + 1) for n > 0. We will 
give another proof of Theorem 3.1 later. 

4 A conjecture of Niederhausen and Sullivan 

A generalized Dyck path is a path starting at (0, 0), with up (/^) steps (1,1) and down (\) 
steps (1, —1), that never goes below the x-axis. Now let us count the number of generalized 
Dyck paths without four consecutive up {/^) steps. Define S{m, n) to be the number of such 
generalized Dyck paths from (0,0) to {m,n) that end with a down (\) step (1, —1). Since 
a path starts at (0, 0), it is natural to set the initial condition S'(0, 0) = 1. It is obvious that 
S{2,0) = 1,S'(3, 1) = 1, and S'(4, 2) = 1. But we have that S'(5, 3) = because we don't 
allow four consecutive up steps. Since a path cannot have four consecutive up (/^) steps, 
such a generalized Dyck path ending at (m, n) can be obtained uniquely from a generalized 
Dyck path ending at {m — i — l,n — i + l) followed by i consecutive up {/^) steps for < i < 3 
and a down (\) step. So, when we define S{m, n) = for n < we have the recurrence 
relation 

2 

S{m, n) = 2^ S{m — i — 2,n — i) for m,n E N, {m, n) ^ (0, 0). 
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Table 5: The values of ^(m, n) 
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Table 6: The values of S'{m,n) 

Table 5 shows the values of S{m, n) with the O's omitted. 

In the paper [1], Niederhausen and Sullivan conjectured that the number S'{m,n) for 
m, n G N defined by 



m,n=0 

is equal to S{4m — n, 2?7i — n) in the case 2m > n. That is, in the case 2m > n > 0, 
S'{m,, n) is the number of generalized Dyck paths from (0, 0) to (4??7, — n, 2m — n) without 
four consecutive up steps that end with a down (\) step (1,-1). Note that there is no 
combinatorial meaning for S'{m,n) when 2m < n. Table 6 shows the values of S'{m,,n). 

5 Main Theorem 

Now we will prove a generalization of the Niederhausen and Sullivan conjecture involving 
more general paths. To do this, let us define a step set T to be a subset of the set N U { — 1}. 
We assume that the step set T has —1 as an element and that K is the largest element of 
T. Now let us consider a path from (0,0) to {m,,0) (where m G N), with steps {l,i) (where 
i G T), that never goes below the x-axis. Put a weight of CjX on each step (1,?), where c, is 
an arbitrary weight for alH G T except c„i = 1 and Cx = 1, and let f{x) be the generating 
function for weighted paths from (0,0) to (m,0), with steps (1,^) (where i G T), that never 
go below the x-axis. 



9 



Figure 2: Decomposition for a path from (0, 0) to {m, 0) when T = { — 1, 2} 

As shown in Figure 2, any such path starting with a step (l,i), where i > 0, can be 
decomposed uniquely [3, pp. 1027-1030] by cutting it right before the first points at height 
j ioT j = i — l,i — 2, . . . ,0. This gives the functional equation for /: 



fix) = ^Ci{xf{x)) 



i+1 



(13) 



ieT 



where the term for i = — 1 corresponds to the empty path. 

Now let us consider a path Q from (0, /i) to (m, 0) (where h,m E N), with steps (1,2) 
(where i & T), that never goes below the x-axis. Decomposing the path Q, as before, into 
the first time it reaches height i for z = /i — 1, /i — 2, . . . , 0, we can deduce that the generating 
function for such weighted paths from (0, h) to {m, 0) is x^f{x)^^^. 

For m, n > and /i > 0, we define Ph{tTi, n) to be the sum of the weights of lattice paths 
from (0, h) to (m, n), with steps (1, i) where i &T, that never go below the x-axis. Then we 
have the initial condition Ph{0,h) = 1 and Ph{m,n) = for n < because paths cannot go 
below the x-axis. Since such paths have steps (1,^) (where z G T) we have the recurrence 



Ph{rn + 1, ra) = 2_. CiPh{fn^ n — i) for m, n > 0. 
Then we can easily deduce 

oo oo 

x^f{xf^^ = J2 Phim, 0)x'" and f{x) = ^ Poim, 0)x'". 



(14) 



m=0 



m=0 



Also, we find the generating function for Ph{m, n): 



J2 Ph{m,n)x"'r = - 



m,n=0 



E 



ieT 



[CiPx) 



E, 



eT 



, CiPx) 



(15) 



where the first term on the right side in (15) counts the sum of the weights of all lattice 
paths start at (0, /i), and the second term counts the sum of the weights of all lattice paths 
start at (0, h) and go below the x-axis. 

Now we want to extend the region of definition of Phim^n) to all integer values of n. 
That is, for m > and n < 0, we want the recurrence (14) to hold below the x-axis. So, for 
m > let us define Pl^im, n) to be Ph{iTi, n) for n > —K, and by 



Ph{m,n) 



Pi{m + 1, n + K) - Y^ c^Piim, n + K 

i£T-{K} 



for n < —K. 



(16) 



10 



4 








2 


4 


12 


32 


72 


194 


3 





1 


1 


6 


9 


27 


60 


137 


2 





1 


2 


3 


10 


16 


44 


93 


1 


1 





2 


2 


5 


12 


21 


56 








1 





2 


2 


5 


12 


21 


-1 


























-2 


























-3 





-1 





-2 


-2 


-5 


-12 


-21 


-4 





1 





2 


2 


5 


12 


21 


-5 


-1 


-1 


-2 


-4 


-7 


-17 


-33 


-77 


-6 


2 


2 


4 


8 


14 


34 


66 


154 


-7 


-3 


-5 


-8 


-17 


-33 


-72 


-155 


-345 


-8 


6 


10 


18 


35 


74 


155 


342 


762 


-9 


-13 


-20 


-39 


-76 


-160 


-344 


-753 


-1696 


-10 


26 


43 


82 


167 


348 


758 


1670 


3759 


njvti 





1 


2 


3 


4 


5 


6 


7 



Table 7: The values of Pl{m, n) with r = { — 1,1,2} 



Note that -P^(Tn, n) in (16) holds for all ?7i G N and ?i G Z. 

Table 7 shows the values of P[{m^ n) with T = { — 1,1,2} and Cj = 1 for all i &T. In this 
table once we know the values of P^(0,n) in the first column, we can figure out the values 
of P{{m,n) for m > by the recurrence that corresponds to the step set T = { — 1, 1,2}. 
For example, P[{l,n) can be computed by summing P[{0,n — i) for all i E T. Similarly, 
P[{m + l,n) can be computed by summing P[{m,n — i) for alH G T and ttt, > 0. So, the 
values of P'^{m,n) are determined by the values of P^(0,ra). Therefore, it is important to 
find out the generating function for P^(0,n) to know the values of P[{m,n). 

In the case T = {—1,0,1,2} with q = 1 for all i E T, we will prove PQ{m,n) = 
S(2m + n, n) (where m,n E N) which is related to the Niederhausen and Sullivan conjecture. 
Now we present a direct way to see the relation between the numbers S{m,n) and PQ{m,n) 
by giving a bijection between the set A of generalized Dyck paths from (0, 0) to {2m + n, n) 
(where n > 0) with no four consecutive up steps that end a down step and the set B of paths 
from (0,0) to {m,n) with steps (1,-1), (1,0), (1,1), and (1,2) that never go below the x- 
axis. Then the number PQ{m^n) becomes the cardinality of the set B. Then the following 
lemma gives us the bijection between the sets A and B. 

Lemma 5.1. For m, n G N, the number S{2m + n, n) is equal to the number PQ{m, n). 

Proof. Let us be given a generalized Dyck path ending at {2m + n, n) that ends a down (\) 
step (1, —1). Then since an up step {/^) of the path is (1, 1) and a down step (\) is (1, —1), 
we can compute that the path has ttt, + n up steps and m down steps. 

Then it is enough to prove this lemma by presenting a bijection from the set A to the 
set B. Now let us decompose the path at every down step of the path. Then we have the 
following four kinds of subpaths: \, /^\, /^/^\, and /^/^/^\. Now let us convert each 
subpath \ to a (1, —1) step, /^\ to a (1, 0) step, /^/^\ to a (1, 1) step, and /^/^/^\ to 
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a (1,2) step. Then we can get a path with steps (1, —1), (1,0), (1, 1), and (1,2) that never 
goes below the x-axis. Similarly, when we are given a path with steps (1, —1), (1, 0), (1,1), 
and (1, 2) that never goes below the x-axis, we can have a generalized Dyck path ending at 
height at least by converting each (1, —1) step to \, each (1,0) step to /^\, each (1, 1) 
step to /"/"%, and each (1, 2) step to /^/^/'\. D 

From the recurrence (16) for P'^{m^n), we have the following lemma. 

Lemma 5.2. Let T C N U { — 1} he a step set and let K he the largest element of T . We 
assume that —1 G T, K ^ —1, ck = 1, and c„i = 1. Then we have for h,m G N, 

Proof. By the recurrence (16) we have 

p;^(m,-K-l) = PUm+l,-l)- Yl c,P;,{m,-l-z) 

ieT-{K} 

= P^(m + l,-l)-P^(m,0) 
= -^.(^,0), 

where the second and third equalities hold because Pl^{m,n) = for —K <n< —1. D 

Now we are trying to find the sum of the weights of the paths with a given step set T. 
For any h,K &N, let us define A{x) to be the generating function for —Plj^{m, 0): 

oo 

A{x) = -J2 Kim, 0)x"^ = -x'f{x)'+\ (17) 

m=0 

and, for j G Z, let Aj{x) be the generating function for Ph{m, —K — 1 — j): 

oo 

A,{x) = Y,P'H{m,-K-l-j)x^. 



m=0 



Then Aj{x) = for —K < j < —1 since Pl^{m,n) = for —K < n < —1, and by Lemma 
5.2 we know Ao{x) = A{x) and A^k~i{x) = —A{x). 

Multiplying both sides of the recurrence (16) by x"^ and summing on m gives 



oo oo 



J2PLim,n)x"' = J2PLim + l,n + K)x"'- ^ CiJ2K{m,n + K-i)x"'. (18) 

m=0 m=0 i^T-{K} m=0 

By setting n = —K — 1 — j, where j > 0, in identity (18) we have 

Ajix) = -{Aj^k{x) - Aj^KiO)) - Y. CiA+,.Kix). (19) 
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Let A{x,t) := Yl^o-^ji^)^'^ ■ Multiplying both sides of identity (19) by P and summing on 
j gives 



0<j<K-l 0<j<K-l 

.7=0 ^ ipT-{K\ \<i<K-i-l ' 



The fifth term on the right side in (20) is A[x) from the following simplification: 

i<=.T-{K} 0<j<K~i~l 0<j<K 

i€T-{K-l} 0<j<K-i-l 
0<j<K 

=A{x), 

where the second and third equalities hold because Aj{x) = for —K < j < — 1 and 
A^K-i{x) = —A{x). 

Therefore, collecting terms in (20) gives 

A{x, t)(^l + Yl ^^^"^"0 = '^(^) Y M^W- (21) 

ieT-{K} j=o 

To find the generating function for P^(0,n), we need to determine A(0,t). To do this, 
we work in the ring of Laurent series C((x))[[t]]. We cannot set x = directly in (21) but 
we can use the following lemma to find the constant term in x. Let us define CT^. to be the 
constant term of a power series in x. 

Lemma 5.3. Let 'H{x) be a power series in x. Let a and (5 he power series in t with 
no constant term. Then in C((x))[[t]] the constant term in x of'H{x)/{l — ax~^ — f3) is 

n{a/ii-p))/ii-^). 

Proof. Let 'H(x) = Yl,'^=o ^nx"' . Then 



l-ax-i-/3 "(i_/3)(i_aa;-V(l-/3)) 



l-P •■ (1 - ai-V(l - /?)) 

^ oo 
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x""^ 



-. oo oo 



X 



/3 ^^ " ^\l-(3 

^ n=0 m=0 

-Hz ^ . D 



1-/3 V1-/3 

Now let us find the constant term in x in A{x,t) by Lemma 5.3. Dividing identity (21) 
by 1 - t^/x + J2ieT~{K} Ci^^"^ gives 

A(x, t) = ^, -^^"^ -^ . (22) 

Since the second term on the right-hand side of (22) contains only negative powers of x, we 
get the constant term in x in A(x, t) as follows: 

A{x) 



CT,,A(a;,t) = CT, 



1 X + z^ieT-{K} ^i^ 

1 

■A 



t^ 



(1 + Z^jeT-IA'}"^*^ ~*) "^ ^ 1 + Z/jer-{K} '^«^ ~*^ 

where the second equality follows from Lemma 5.3 and the last equality follows from (17). 

To simplify the constant term A(0,t) that we computed above, let us define a power 
series g by 

Then we have 



9{t) = mKm))' ^^''' ^^^^ = E^^^"""- (23) 



j=0 

oo 

= J2PL{0,-K-l-j)t^. 
Since -P^(0, n) = for n > —K except for P[^{0, h) = 1, we have 

neZ nGN 

Finally, from the above generating function for P^(0, h — n), we can find the generating 
function for P'f^{m^n). To do this, we rewrite the recurrence (16) as 



Pl^{m + l,n + K) = y^CiPl^{m,n + K -i). 
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Then replacing n with Km + h — n gives 

P'h{m + l,K{m + l) + h-n) = ^CiP!^{m, K{m + l) + h-n-i). 

Next, multiplying both sides by t" and summing on n gives 

oo oo 

^Pl^{m+ 1, K{m + l) + h-n)r = B{t) ^ Pl^{m, Km + h - n)t". 

n=0 n=0 

Finally, we conclude 



J2 Ki^, Km + h- n)r = Bit)"" J^ ^h^^^ ^ ' ^)^" 

n=0 



n=0 n=0 

\h+l,{h+l){K+l) 



We summarize with the following theorem. 

Theorem 5.1. Let T G N U { — 1} be a step set and let K be the largest element of T . We 
assume that — 1 G T and K ^ —1. Let Ph{m, n) be the sum of the weights of the paths from 
{0,h) to {m,n) (where h,m,n G N), with steps (l,i) (where i G T), that never go below the 
X-axis, where we put a weight of CiX on each step (1,«) (where Ci is an arbitrary weight for 
all i ^ T except c„i = 1 and ck = i)- For each ?72 > 0, define P^^m,, n) for n < Km + h by 



Vh{x, t)= Y^ Pi{m, Km + h- n)x"'t'' 

m, n=0 

\ _ g^tf + lt^h + im + l)-^ (^^ _ ^^^^) 



where the power series g{t) is uniquely determined by 

y: c.t^-9{t)j2{t''^'9it)r'=h (24) 

iGT-{0,-l} n=l 

and Pfiim,, n) = for n > Km + h. Then we have 

Ph{m, n) = Ph{m, n) for m,n > 0. 

Proof. It is enough to show that the power series g defined by (23) satisfies the functional 
equation (24) and g is uniquely determined by (24). From the functional equation (13) for 
/, we can find the functional equation for g. Substituting t^ /B{t) for x in equation (13) and 
simplifying gives the functional equation for g 



B{t)g{t) = Yc,{t^9ii)Y^'- (25) 
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Since B{t) = 2, Cit * where c_i = 1 and Ck = I, our theorem follows from the factorization: 

ieT-{o,-i} ^ ^ ieT-{o,-i} ^ 

jeT-{o,-i} ieT-{o,-i} 

^ ieT-{0,-l} n=l ^ 

Now we want to show that the power series g is uniquely determined by (24). Every 
term, except i = K and ra = 1, on the left side of (24) has a factor P for j > 1 and the term 
for i = K and ra = 1 is g{t). Then the functional equation (24) may be written as 

g{t) = 1 + tT(t,g(t)), for some polynomial T in t. 

Equating coefficients gives a unique power series solution. Therefore the power series g is 
uniquely determined by (24). D 

Note that in Theorem 5.1 we prefer to take Pl^{m,Km + h — n) instead oi Pl^{m,n) as the 
coefficient of x"^t"' in Vh{x,t) because we want Vh{x,t) to be a power series, not a Laurent 
series. Comparing the generating function (15) for Ph{m, n) with the generating function for 
Pl^{m,n) in Theorem 5.1, we can see that the degree of equation (24) that g satisfies is one 
less than the degree of equation (13) that / satisfies. In particular, when the largest element 
of the step set T is 2 (that is, K = 2) we know that the degree of / is 3 and the degree of 
the corresponding function g is 2. So, while the power series g can be easily computed by 
using the quadratic formula, the power series / cannot be. We will see more details with 
examples in the following subsection. 

5.1 Corollaries to our main theorem 

First let us consider the case T = {—1, 1} and h = 0. Then we will see that Theorem 5.1 
gives (8) in this case. Since we defined B{m, n) to be the number of lattice paths from (1, 0) 
to (m, n) (where m > n), with unit up steps (0, 1) and unit right steps (1,0), that stay below 
the line x = y we can conclude 

B{m + l,n) = PQ^m + n,m — n) for m > n. 

Since the functional equation for g in Theorem 5.1 is g{t) = 1, we have 



v^ „.. . ™.„ 1-t' 



E ^o("^, rn - n)x-r = ^_ ■ (26) 

m,n=0 ^ ^ 
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Replacing t^ with t and n with 2n in (26) gives 

°° ] -t 

J2 Poi^. rn - 2n)x-r = ^_ ^^ (27) 

m,n=0 ^ ' 

Next, replacing m with ?7i + n and t with tjx in (27) gives 

Y^ P',{m + n,m- n)x-e = ^^_~^_ (28) 

n.>0 ^ ' 

m>—n 

Since the right side of equation (28) does not have terms x^P for i < — 1, we reduce the 
range from m > —n to rra > — 1 on the left side. That is, 

V P'im + n,m- n)x"'e = , ^~^ , . (29) 

n.>0 ^ ' 

m> — 1 

Finally, multiplying by x and substituting m + 1 for m in (29) gives 

V P'im - 1 + n, m - 1 - n)x'"r = — , 

^^^ ^ \- x-t 

m,n=0 

which is the same as (8) because 

B^m + 1, n) = Po(m + n,m — n) and B^m, n) = B{m^ n) for < n < m. 

A more interesting case is T = {—1,0, 1,2} where K = 2. As we mentioned before, in 
this case we can easily compute the power series g using the quadratic formula. Then the 
functional equation for g in Theorem 5.1 becomes t^g{tY + (1 + cit)g{t) — 1 = 0, so 



-1 - Cit + VI + 2Cit + Ci2t2 + 4t3 

9if) = ^^ , (30) 

where the sign of the square root is determined because g is a power series. 

Now as a special case we are going to prove the Niederhausen and Sullivan conjecture 
and give another proof of Theorem 3.1. For the conjecture we set /i = 0, Cq = 1, and Ci = 1 
because the paths start at (0, 0) and each path with steps (1, i) where i E T = { — 1, 0, 1, 2} 
is counted once. Then by (30) we have the power series 



,, -1 - t + VI + 2t + t2 + 4t3 
'^'^ = 2t^ • 

That is, g{t) = l-t + t^-2t'^ + U^-7t^ + 13t'^-26f + 52t^-104:t^ + 212t^'^ + 
So, by Theorem 5.1 we have the generating function for the values of PQ{m,n) 



J: P^im,2m-n)x-r='-±^-yi±f±^±^{l-xil + t + f + t^)} 



3 + t - VI + 2t + t2 + 4t3 
t^Q^rn, zm — njx'i" = 

m,n=0 

17 
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13 
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-14 
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6 


7 



Table 8: The values of PQ{m,n) 

which is the conjecture [1] of Niederhausen and Sullivan because S'{m,n) = PQ(m, 2m — n) 
for m,n > 0. By Pq(?ti, 2?72 — n) = 3(4:771 — n, 2m — n) where 2m > n, we conclude 
S'{7n,7i) = S{A77i — 7i,27n — n) for < n < 27n. That is, S'{77i,7i) is the number of paths 
from (0, 0) to (4m — n— 1, 2m — n + 1), with up (/^) steps (1, 1) and down (\) steps (1, —1), 
that avoid four consecutive up {/^) steps and never go below the x-axis. 

For another proof of Theorem 3.1, first let us consider a lattice path R from (0,0) to 
{7n,7i) (where n < 2m), with unit up (t) steps (0,1) and unit right (— )■) steps (1,0), that 
never crosses the line y = 2x. To apply Theorem 5.1 we transform the path R into a path 
from (0, 0) to (m + 7i, 2m — n), with up steps (1, 2) and down steps (1, —1), that never goes 
below the a;-axis. We replace a unit right (— )■) step (1, 0) with a step (1, 2) and a unit up (t) 
step (0, 1) with a step (1, —1). The transformed path does not go below the x-axis because 
when the path R takes a unit right (— )■) step (1, 0) the y-coordinate difference between the 
path R and the line y = 2x increases by 2, whereas when the path R takes a up (t) step 
(0, 1) the y-coordinate difference decreases by 1. So, we have 



02(771, n) = PQ{m + 71, 2m — n) for 2m, > n. 



(31) 



In Theorem 5.1 we take the step set T = { — 1, 2}. Then we set /i = 0, cq = 0, and ci = 
since paths start at (0,0) and have the step set T = { — 1,2}. Table 8 shows the values of 
PQ(m, n). 

We see from Table 4 and Table 8 that 



Do 



m,n] 



Poim 



71, 2m — n) for n > and m + n > 0. 



(32) 
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We can prove that (32) holds by using the recurrences (12) and (16). There is no combina- 
torial meaning for D2{m, n) where 2?7i < n or n < 0. 
By (30) we have the power series 



n=0 

That is, g{t) = 1 - t^ + 2t^ - 5t^ + Ut^"^ - i2t^^ + 132t^^ - 429t^^ + U30t^^ + ■ ■ ■ . So, by 
Theorem 5.1 the generating function for the values of PQ{m,n) is 



oo oo 

Y^ P;,{m, 2m - n)x"'r = (l + Y,i-'^T''^CJ^^"^'^) (l - <'^ + ^'^ 



-1 



m,n=0 n=0 

Replacing t with t^'^ and 3n with n gives 



oo oo 

Y^ Pl,{m, 2m - 3n)x"'r = (l + Y^-lT^^Cnt''^') (l - x(l + t) 



m,n=0 n=0 



With D2{m,, n) defined as in (1), we know Po(m, 2m — 3n) = Di^im, — n, n) for m,n ^N. So, 
we have 

D'^i-n - 1, n + 1) = (-1)"+^C„ for all n>0. 

From (32) we conclude 

D2(-n - 1, n + 1) = (-1)"+^C„ for all n > 0. 



Also, by (31) we deduce 



D'olm, n) = D2(m,, n) ioT < n < 2m. 



That is, D'2{m,n) is the number of lattice paths from (0,0) to {m,,n) (where < n < 2m), 
with unit up (t) steps (0, 1) and unit right (— )■) steps (1, 0), that never cross the line y = 2x. 
For another application of Theorem 5.1 we can find a redundant generating function for 
Dp{m, n) which was defined in Section 3. To do this let us consider a lattice path S from 
(0, 0) to (m, n) (where n < pm), with unit up (f) steps (0, 1) and unit right (— )■) steps (1,0), 
that never crosses the line y = px. To apply Theorem 5.1 we transform the path S into 
a path from (0,0) to (m + n,pm, — n), with up steps (l,p) and down steps (1,-1), that 
never goes below the x-axis. We replace a unit right (— )■) step (1,0) with a step (l,p) and 
a unit up (t) step (0, 1) with a step (1,-1). The transformed path does not go below the 
X-axis because when the path 5* takes a unit right (— )>) step (1, 0) the y-coordinate difference 
between the path S and the line y = px increases by p, whereas when the path S takes a up 
(t) step (0, 1) the ^/-coordinate difference decreases by 1. So we take the step set T = {—l,p} 
and Dp{m, n) = Po(m -(- n,pm, — n). By Theorem 5.1, we know that the functional equation 
for g{t) is 



P ■ ■ n~l 



git) E (aitrn 



n=l 
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which is equal to Yln=i idi^)^^^^) — t^~^^- From this equation we see that g{t)t'^^^ is a 
power series in t^+-^, so we may define a power series 7(t) by ^{t^^^) = g{t)t'^^^ . Then the 
equation becomes 

J2l{tT = t. (33) 

n=l 

That is, 7(t) is the compositional inverse of t + t^ + ■ ■ ■ + t^. So, by Theorem 5.1 (where 
/i = 0) we have the generating function for the values of -Po(?7i, n) is 

oo 

Y^ P^(m,j9m-(|)+l)n)x™r= (l-^(t)tP+^)(l-x(l+tP+^))~\ (34) 

m,n=0 

By Dp(m — n,n) = PQ(^m,pm — {p+l)n) and Theorem 5.1 (where /i = 0), we can deduce 
the following corollary after replacing t^^^ with t and (p + l)n with n in (34). 

Corollary 5.1. Let Dp{'m,n) be the number of lattice paths from (0,0) to {m,n) (where 
n < pm), with unit up (t) steps (0, 1) and unit right (— >) steps (1,0), that never cross the 
line y = px. Define the number D' Jm, n) for m,n &N by 



-1 



J2 D'p{m - n, n)x"'t'' = (l - l{t)) (l - a;(l + t 

m, n=0 

where the power series 7(t) satisfies equation (33). Then we have 

D' {m, n) = Dp{m, n) for < n < pm. 

Up to now we considered a path starting at height 0, that is, h = 0. Now we want 
to consider the more general case of a boundary line y = px + h instead oi y = px where 
/i,p G P. For a fixed h,p E F, let £'p^/i(?n,?T,) be the number of lattice paths from (0,0) to 
(m, n) (where n < pm + h), with unit up (f) steps (0, 1) and unit right (— )■) steps (1, 0), that 
never cross the line y = px + h. 

Applying the same transformation as before, we have 

Ep^hiji^ — n,n) = Pl^(m,pm + h — {p + l)n^ with T = { — l,p}. 

Therefore, by Theorem 5.1 we can conclude the following corollary after replacing t^"*"^ with 
t and {p + l)n with n. 

Corollary 5.2. For h,p eN, the number E' ,(m,n) for m,n eN is defined by 

oo 

J2 E'p,h^rn - n, n)x^t- = (l - ^{tf^') (l - x{l + t)Y\ 

m, 71=0 

where the power series 7(t) satisfies equation (33). Then we have 

E' f^{m, n) = Eph{m, n) for < n < pm + h. 

Note that in the case h = 0, Corollary 5.2 reduces to Corollary 5.1 because Epfi{m,n) = 
Dp{m,n). 
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